Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations.

Background: Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. Methods: We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1-4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models' predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models' forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models' past predictive performance. Results: Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models' forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models' forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models' forecasts of deaths (N=763 predictions from 20 models). Across a 1-4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. Conclusions: Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks. Funding: AA, BH, BL, LWa, MMa, PP, SV funded by National Institutes of Health (NIH) Grant 1R01GM109718, NSF BIG DATA Grant IIS-1633028, NSF Grant No.: OAC-1916805, NSF Expeditions in Computing Grant CCF-1918656, CCF-1917819, NSF RAPID CNS-2028004, NSF RAPID OAC-2027541, US Centers for Disease Control and Prevention 75D30119C05935, a grant from Google, University of Virginia Strategic Investment Fund award number SIF160, Defense Threat Reduction Agency (DTRA) under Contract No. HDTRA1-19-D-0007, and respectively Virginia Dept of Health Grant VDH-21-501-0141, VDH-21-501-0143, VDH-21-501-0147, VDH-21-501-0145, VDH-21-501-0146, VDH-21-501-0142, VDH-21-501-0148. AF, AMa, GL funded by SMIGE - Modelli statistici inferenziali per governare l'epidemia, FISR 2020-Covid-19 I Fase, FISR2020IP-00156, Codice Progetto: PRJ-0695. AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by Ministry of Science and Higher Education of Poland with grant 28/WFSN/2021 to the University of Warsaw. BRe, CPe, JLAz funded by Ministerio de Sanidad/ISCIII. BT, PG funded by PERISCOPE European H2020 project, contract number 101016233. CP, DL, EA, MC, SA funded by European Commission - Directorate-General for Communications Networks, Content and Technology through the contract LC-01485746, and Ministerio de Ciencia, Innovacion y Universidades and FEDER, with the project PGC2018-095456-B-I00. DE., MGu funded by Spanish Ministry of Health / REACT-UE (FEDER). DO, GF, IMi, LC funded by Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL) under project number 20200700ER. DS, ELR, GG, NGR, NW, YW funded by National Institutes of General Medical Sciences (R35GM119582; the content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS or the National Institutes of Health). FB, FP funded by InPresa, Lombardy Region, Italy. HG, KS funded by European Centre for Disease Prevention and Control. IV funded by Agencia de Qualitat i Avaluacio Sanitaries de Catalunya (AQuAS) through contract 2021-021OE. JDe, SMo, VP funded by Netzwerk Universitatsmedizin (NUM) project egePan (01KX2021). JPB, SH, TH funded by Federal Ministry of Education and Research (BMBF; grant 05M18SIA). KH, MSc, YKh funded by Project SaxoCOV, funded by the German Free State of Saxony. Presentation of data, model results and simulations also funded by the NFDI4Health Task Force COVID-19 (https://www.nfdi4health.de/task-force-covid-19-2) within the framework of a DFG-project (LO-342/17-1). LP, VE funded by Mathematical and Statistical modelling project (MUNI/A/1615/2020), Online platform for real-time monitoring, analysis and management of epidemic situations (MUNI/11/02202001/2020); VE also supported by RECETOX research infrastructure (Ministry of Education, Youth and Sports of the Czech Republic: LM2018121), the CETOCOEN EXCELLENCE (CZ.02.1.01/0.0/0.0/17-043/0009632), RECETOX RI project (CZ.02.1.01/0.0/0.0/16-013/0001761). NIB funded by Health Protection Research Unit (grant code NIHR200908). SAb, SF funded by Wellcome Trust (210758/Z/18/Z).

Implementing the Lolli-Method and pooled RT-qPCR testing for SARS-CoV-2 surveillance in schools: a pilot project.

PURPOSE: School closures have been used as part of lockdown strategies to contain the spread of SARS-CoV-2, adversely affecting children's health and education. To ensure the accessibility of educational institutions without exposing society to the risk of increased transmissions, it is essential to establish SARS-CoV-2 testing strategies that are child-friendly, scalable and implementable in a daily school routine. Self-sampling using non-invasive saliva swabs combined with pooled RT-qPCR testing (Lolli-Method) has been proven to be a sensitive method for the detection of SARS-CoV-2. METHODS: We conducted a pilot project in Cologne, Germany, designed to determine the feasibility of a large-scale rollout of the Lolli-Method for testing without any additional on-site medical staff in schools. Over a period of three weeks, students from 22 schools were sampled using the Lolli-Method. At the end of the project, teachers were asked to evaluate the overall acceptance of the project. RESULTS: We analyzed a total of 757 pooled RT-qPCRs obtained from 8,287 individual swabs and detected 7 SARS-CoV-2 infected individuals. The Lolli-Method was shown to be a feasible and accepted testing strategy whose application is only slightly disruptive to the daily school routine. CONCLUSION: Our observations suggest that the Lolli-Method in combination with pooled RT-qPCR can be implemented for SARS-CoV-2 surveillance in daily school routine, applicable on a large scale.

Age-specific contribution of contacts to transmission of SARS-CoV-2 in Germany.

Current estimates of pandemic SARS-CoV-2 spread in Germany using infectious disease models often do not use age-specific infection parameters and are not always based on age-specific contact matrices of the population. They also do usually not include setting- or pandemic phase-based information from epidemiological studies of reported cases and do not account for age-specific underdetection of reported cases. Here, we report likely pandemic spread using an age-structured model to understand the age- and setting-specific contribution of contacts to transmission during different phases of the COVID-19 pandemic in Germany. We developed a deterministic SEIRS model using a pre-pandemic contact matrix. The model was optimized to fit age-specific SARS-CoV-2 incidences reported by the German National Public Health Institute (Robert Koch Institute), includes information on setting-specific reported cases in schools and integrates age- and pandemic period-specific parameters for underdetection of reported cases deduced from a large population-based seroprevalence studies. Taking age-specific underreporting into account, younger adults and teenagers were identified in the modeling study as relevant contributors to infections during the first three pandemic waves in Germany. For the fifth wave, the Delta to Omicron transition, only age-specific parametrization reproduces the observed relative and absolute increase in pediatric hospitalizations in Germany. Taking into account age-specific underdetection did not change considerably how much contacts in schools contributed to the total burden of infection in the population (up to 12% with open schools under hygiene measures in the third wave). Accounting for the pandemic phase and age-specific underreporting is important to correctly identify those groups of the population in which quarantine, testing, vaccination, and contact-reduction measures are likely to be most effective and efficient. Age-specific parametrization is also highly relevant to generate informative age-specific output for decision makers and resource planers.

Infection and transmission risks of COVID-19 in schools and their contribution to population infections in Germany: A retrospective observational study using nationwide and regional health and education agency notification data.

BACKGROUND: School-level infection control measures in Germany during the early Coronavirus Disease 2019 (COVID-19) pandemic differed across the 16 federal states and lacked a dependable evidence base, with available evidence limited to regional data restricted to short phases of the pandemic. This study aimed to assess the (a) infection risks in students and staff; (b) transmission risks and routes in schools; (c) effects of school-level infection control measures on school and population infection dynamics; and (d) contribution of contacts in schools to population cases. METHODS AND FINDINGS: For this retrospective observational study, we used German federal state (NUTS-2) and county (NUTS-3) data from public health and education agencies from March 2020 to April 2022. We assessed (a) infection risk as cumulative risk and crude risk ratios and (b) secondary attack rates (SARs) with 95% confidence interval (CI). We used (c) multiple regression analysis for the effects of infection control measures such as reduced attendance, mask mandates, and vaccination coverage as absolute reduction in case incidence per 100,000 inhabitants per 14 days and in percentage relative to the population, and (d) infection dynamic modelling to determine the percentage contribution of school contacts to population cases. We included (a) nationwide NUTS-2 data from calendar weeks (W) 46-50/2020 and W08/2021-W15/2022 with 3,521,964 cases in students and 329,283 in teachers; (b) NUTS-3 data from W09-25/2021 with 85,788 student and 9,427 teacher cases; and (c) detailed data from 5 NUTS-3 regions from W09/2020 to W27/2021 with 12,814 cases (39% male, 37% female; median age 14, range 5 to 63), 43,238 contacts and 4,165 secondary cases for students (for teachers, 14,801 [22% male, 50% female; median age 39, range 16 to 75], 5,893 and 472). Infection risk (a) for students and teachers was higher than the population risk in all phases of normal presence class and highest in the early 2022 omicron wave with 30.6% (95% CI 30.5% to 32.6%) of students and 32.7% (95% CI 32.6% to 32.8%) of teachers infected in Germany. SARs (b) for students and staff were below 5% in schools throughout the study period, while SARs in households more than doubled from 13.8% (95% CI 10.6% to 17.6%) W21-39/2020 to 28.7% (95% CI 27% to 30.4%) in W08-23/2021 for students and 10.9% (95% CI 7% to 16.5%) to 32.7% (95% CI 28.2% to 37.6%) for staff. Most contacts were reported for schools, yet most secondary cases originated in households. In schools, staff predominantly infected staff. Mandatory surgical mask wearing during class in all schools was associated with a reduction in the case incidence of students and teachers (c), by 56/100,000 persons per 14 days (students: 95% CI 47.7 to 63.4; teachers: 95% CI 39.6 to 71.6; p < 0.001) and by 29.8% (95% CI 25% to 35%, p < 0.001) and 24.3% (95% CI 13% to 36%, p < 0.001) relative to the population, respectively, as were reduced attendance and higher vaccination coverage. The contribution of contacts in schools to population cases (d) was 2% to 20%, lowest during school closures/vacation and peaked during normal presence class intervals, with the overall peak early during the omicron wave. Limitations include underdetection, misclassification of contacts, interviewer/interviewee dependence of contact-tracing, and lack of individual-level confounding factors in aggregate data regression analysis. CONCLUSION: In this study, we observed that open schools under hygiene measures and testing strategies contributed up to 20% of population infections during the omicron wave early 2022, and as little as 2% during vacations/school closures; about a third of students and teachers were infected during the omicron wave in early 2022 in Germany. Mandatory mask wearing during class in all school types and reduced attendance models were associated with a reduced infection risk in schools.